Although the Internet has had great successes in facilitating communications between computer systems and enabling electronic commerce, the computer systems connected to the Internet have been under almost constant attack by hackers seeking to disrupt their operation. Many of the attacks seek to exploit vulnerabilities of software systems including application programs or other computer programs executing on those computer systems. Developers of software systems and administrators of computer systems of an enterprise go to great effort and expense to identify and remove vulnerabilities. Because of the complexity of software systems, however, it is virtually impossible to identify and remove all vulnerabilities before software systems are released. After a software system is released, developers can become aware of vulnerabilities in various ways. A party with no malicious intent may identify a vulnerability and may secretly notify the developer so the vulnerability can be removed before a hacker identifies and exploits it. If a hacker identifies a vulnerability first, the developer may not learn of the vulnerability until it is exploited—sometimes with disastrous consequences.
Regardless of how a developer finds out about a vulnerability, the developer typically develops and distributes to system administrators “patches” or updates to the software system that remove the vulnerability. If the vulnerability has not yet been exploited (e.g., might not be known to hackers), then a developer can design, implement, test, and distribute a patch in a disciplined way. If the vulnerability has already been widely exposed, then the developer may rush to distribute a patch without the same care that is used under normal circumstances. When patches are distributed to the administrators of the computer systems, they are responsible for scheduling and installing the patches to remove the vulnerabilities.
Unfortunately, administrators often delay the installation of patches to remove vulnerabilities for various reasons. When a patch is installed, the software system and possibly the computer system on which it is executing may need to be shut down and restarted. If the vulnerability is in a software system that is critical to the success of an enterprise, then the administrator needs to analyze the tradeoffs of keeping the software system up and running with its associated risk of being attacked and of shutting down a critical resource of the enterprise to install the patch. Some administrators may delay the installation of the patch because they fear that, because of a hasty distribution, it might not be properly tested and have unintended side effects. If the patch has an unintended side effect, then the software system, the computer system, or some other software component that is impacted by the patch may be shut down by the patch itself. Administrators need to factor in the possibility of an unintended side effect when deciding whether to install a patch. These administrators may delay installing a patch until experience by others indicates that there are no serious unintended side effects.
Intrusion detection systems have been developed that can be used to identify whether an attempt is being made to exploit a known vulnerability that has not yet been patched. These intrusion detection systems can be used to prevent exploitations of newly discovered vulnerabilities for which patches have not yet been developed or installed. These intrusion detection systems may define a “signature” for each way a vulnerability can be exploited. For example, if a vulnerability can be exploited by sending a certain type of message with a certain attribute, then the signature for that exploitation would specify that type and attribute. When a security enforcement event occurs, such as the receipt of a message, the intrusion detection system checks its signatures to determine whether any match the security enforcement event. If so, the intrusion detection system may take action to prevent the exploitation, such as dropping the message.
A set of one or more signatures may be considered a security policy. Developers of intrusion detection systems may provide various security policies. For example, a developer may provide one security policy that defines signatures of vulnerabilities of an operating system and many other security policies that are specific to an application or a class of applications. Similarly, an administrator may define a security policy that is specific to custom applications used by the enterprise.
Because intrusions can occur at various points within an operating system or an application, intrusion detection systems have been developed to detect and prevent exploitation of vulnerabilities at each of these points. For example, an intrusion detection system may prevent exploitation of vulnerabilities that can be detected at the transport layer of a communication protocol. This intrusion detection system needs to intercept communications between a transport layer and a higher layer and determine whether the communications are an attempt to exploit a vulnerability.
Many implementations of transport layers adhere to the Transport Device Interface (“TDI”) of Microsoft Corporation. TDI defines the interactions between a “transport provider” that implements the transport layer and a “transport client” that uses the services of the transport provider. One example of a transport provider is a TCP driver, and one example of a transport client is a redirector of a file system. The redirector invokes functions provided by the TCP driver to send and receive messages via TCP. FIG. 1 is a block diagram illustrating interactions between a TDI transport client and a TDI transport provider. In this example, the transport provider and transport client both execute in kernel mode. The transport client 102 may request services of the transport provider 101 through I/O manager 103. The I/O manager may provide high-level function calls to the transport client and map those high-level function calls to function calls of the transport provider. For example, when the transport client performs function call 110, the I/O manager maps it to function call 111. The functions of the transport provider may be asynchronous in that a function returns before the requested service is completed. Thus, when a function call is made, the transport client provides a completion routine through which the transport provider can asynchronously notify the transport client when the service requested by the function call has been completed by invoking the completion routine 112. If the function call was made via the I/O manager, then the I/O manager invokes the completion routine 113 of the transport client. A transport client may also invoke the functions of the transport provider directly 114. The transport client may also register with the transport provider to be notified of certain events asynchronously. For example, the transport client may want to be notified when a packet is received. The transport client may register a callback for each type of event. When the transport provider detects an event for which the transport client has registered a callback, the transport provider calls the corresponding registered callback 116 to provide event-based notification to the transport client.
Developers of intrusion detection systems may develop many different ways to enforce security policies and at various levels of a communications protocol. It would be desirable to have an effective way to intercept communications at the transport layer and enforce security policies.